Image processing apparatus, method and program

ABSTRACT

An image processing apparatus is connected to and accesses a database that stores items of characteristic quantity information to be used for recognizing, in image data, respective objects and items of color information of the respective objects, the characteristic quantity information and the color information are correlated with each other for the respective objects. A controller performs image recognition processing on image data using the items of characteristic quantity information stored in the database, and acquires color information of an object that has been recognized in the image data by the image recognition processing. The controller searches the database to retrieve the color information indicating a color of the object recognized by the image recognition processing, and identifies a color space of the image data by comparing the acquired color information with the retrieved color information.

This is a Divisional of application Ser. No. 10/725,423 which was filedon Dec. 3, 2003. The disclosure of the prior application is herebyincorporated by reference herein in its entirety.

BACKGROUND OF THE INVENTION

1. Field of Invention

The present invention relates to an image processing apparatus, methodand program which perform prescribed color-related processing on a colorimage.

2. Description of Related Art

Various methods for expressing color image data are available. Forexample, in a method using the RGB color space, the color of each pixelconstituting image data is represented by luminance information of thethree primary colors of red, green, and blue. In another method, thecolor of each pixel is represented by the ink primary colors of cyan(C), magenta (M), yellow (Y), and black (K). In such colorrepresentation methods that are generally device-dependent, colorshaving the same RGB values are seen as different colors according to thehuman visual sense depending on the color coordinates of primary colorsused, the white point definition, etc. For example, the standard knownas adobeRGB has a wider color range than the standard known as sRGB,which was established to serve as an HDTV color standard, and thereforecolors having the same RGB values are seen differently according tothese two standards. Similarly, a color according to the print colorsamples of JapanColor and a color according to those of SWOP, which is aU.S. print standard, are seen (appear) differently even if they have thesame CMYK values.

In recent years, to eliminate the above-described non-coincidence incolor representation, image data formats have been developed in whichimage data contains information (color space identification information)indicating a color space that is used for expressing the image data.Specifically, a data format called “ICC (International Color Consortium)profile” and that includes color information is known.

However, the present situation is such that much of the conventionalimage data that are generated and used in many devices do not containsuch color space identification information. For this reason, a methodof using a pre-assumed color space and a method of causing a user toinput color space identification information of image data are widelyused when it is necessary to identify a color space to, for example,change color spaces. However, in the method of using a pre-assumed colorspace, a desired processing result is not obtained if a color spaceactually employed is different from the assumed one.

The method of causing a user to input color space identificationinformation has a problem in that it is less convenient because the userneeds to check a color space.

Further, even if image data contains color space identificationinformation, there is a possibility that it is in error. Thus, there isa need for a method for checking the legitimacy of color spaceidentification information contained in image data.

SUMMARY OF THE INVENTION

The present invention has been made in view of the above circumstances,and provides an image processing apparatus, method and program capableof easily generating and checking color space identification informationrelating to image data that is to be processed.

To address the problems in the art, one aspect of the invention providesan image processing apparatus that accesses a database that stores (1)items of characteristic quantity information to be used for recognizing,in image data, respective objects and (2) items of color information ofthe respective objects, the characteristic quantity information and thecolor information are correlated with each other for the respectiveobjects. The image processing apparatus includes a controller that: (1)performs image recognition processing on image data using the items ofcharacteristic quantity information stored in the database and acquirescolor information of an object that has been recognized in the imagedata by the image recognition processing; and (2) searches the databaseto retrieve the color information indicating a color of the objectrecognized by the image recognition processing and identifies a colorspace of the image data by comparing the acquired color information withthe retrieved color information.

Another aspect of the invention provides an image processing apparatuswhich accesses a database that stores (1) items of characteristicquantity information to be used for recognizing, in image data,respective objects as subjects and (2) items of color information of therespective objects, the characteristic quantity information and thecolor information are correlated with each other for the respectiveobjects. The image processing apparatus includes a controller that (1)judges whether information indicating a color space of image data, andwhich is appended to and input together with the image data, satisfies aprescribed condition; (2) performs image recognition processing on theimage data using the items of characteristic quantity information storedin the database and acquires color information of an object that hasbeen recognized in the image data by the image recognition processing,if it is judged that the information indicating the color space and thatis appended to the image data does not satisfy the prescribed condition;and (3) searches the database to retrieve the color informationindicating a color of the object recognized by the image recognitionprocessing and identifies a color space of the image data by comparingthe acquired color information with the retrieved color information.

It is preferable that the controller of the image processing apparatusfurther performs statistical processing on identification results ofcolor spaces of the image data that was previously processed; andperforms prescribed processing using a result of the statisticalprocessing.

Another aspect of the invention relates to an image processing methodusing an image processing apparatus which accesses a database thatstores (1) items of characteristic quantity information to be used forrecognizing, in image data, respective objects and (2) items of colorinformation of the respective objects, the characteristic quantityinformation and the color information are correlated with each other forthe respective objects. The method includes the steps of: (1) performingimage recognition processing on image data using the items ofcharacteristic quantity information stored in the database, andacquiring color information of an object that has been recognized in theimage data by the image recognition processing; and (2) searching thedatabase to retrieve the color information indicating a color of theobject recognized by the image recognition processing, and identifying acolor space of the image data by comparing the acquired colorinformation with the retrieved color information.

Another aspect of the invention relates to an image processing programfor causing a computer which accesses a database that stores (1) itemsof characteristic quantity information to be used for recognizing, inimage data, respective objects and (2) items of color information of therespective objects, the characteristic quantity information and thecolor information are correlated with each other for the respectiveobjects, to execute the steps of: (1) performing image recognitionprocessing on image data using the items of characteristic quantityinformation stored in the database, and acquiring color information ofan object that has been recognized in the image data by the imagerecognition processing; and (2) searching the database to retrieve thecolor information indicating a color of the object recognized by theimage recognition processing, and identifying a color space of the imagedata by comparing the acquired color information with the retrievedcolor information.

BRIEF DESCRIPTION OF THE DRAWINGS

Preferred exemplary embodiments of the present invention will bedescribed in detail based on the following figures, in which:

FIG. 1 is a block diagram showing the configuration of an imageprocessing apparatus according to a first embodiment of the presentinvention;

FIG. 2 is a functional block diagram of a control section (controller)11 of the image processing apparatus of FIG. 1;

FIG. 3 is a functional block diagram of an example of a reliabilityjudging section 23;

FIG. 4 outlines an example of the contents of an object database;

FIG. 5 is a functional block diagram of an example of a color spaceinferring section 26;

FIG. 6 is a flowchart showing an exemplary operation of the imageprocessing apparatus of FIG. 1;

FIG. 7 is a functional block diagram of a control section (controller)11 of an image processing apparatus according to a second embodiment ofthe invention; and

FIG. 8 is a flowchart showing an exemplary operation of the imageprocessing apparatus according to the second embodiment.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

A first embodiment of the present invention will be describedhereinafter with reference to the drawings. As shown in FIG. 1, an imageprocessing apparatus according to this embodiment is composed of acontrol section (controller) 11, a storage section 12, an input/outputinterface section 13, a manipulation section 14, and a display section15.

The control section 11, which operates according to programs stored inthe storage section 12, performs prescribed processing on image data asa subject of processing that is input through the input/output interfacesection 13, and outputs processed data to an external device through theinput/output interface section 13. The details of processing performedby the control section 11 will be described later in detail. The storagesection 12 includes a storage element such as, for example, a RAM and adisk device such as a hard disk drive. The storage section 12 operatesas a computer-readable recording medium.

The input/output interface section 13 receives image data from anexternal device and supplies it to the control section 11. Further, theinput/output interface section 13 outputs, to an external device,processed image data that is supplied from the control section 11. Themanipulation section 14, which is, for example, a keyboard, a mouse,etc., supplies information indicating the contents of a manipulation bya user to the control section 11. The display section 15, which is, forexample, a CRT display, an LCD display, or the like, presents image datato a user according to an instruction from the control section 11.

The details of processing performed by the control section 11 will bedescribed below. As shown in FIG. 2, programs that are executed by thecontrol section 11 include, in terms of functions, an acceptance section21, a color space inferring section 26, a reliability judging section23, an image processing section 24, and an output section 25. It isassumed that data to be processed by the control section 11 has astructure that is prescribed by Exif (exchangeable image file format fordigital still cameras). This data structure will not be described indetail because it is widely publicized by Japan Electronic IndustryDevelopment Association. The Exif data structure has appendedinformation that includes various tags relating to (i.e., identifying)information. These tags include a tag relating to a version (e.g., aversion of the Exif data structure (Exif Version)), a tag relating to animage data characteristic (color space information (ColorSpace)), a tagrelating to structure (e.g., image compression mode(CompressedBitsPerPixel)), a tag relating to user information, a tagrelating to related file information, a tag relating to a date and time(a date and time of generation of original image data (e.g.,DateTimeOriginal) and a date and time of generation of digital data(e.g., DataTimeDigitized)), and a tag relating to shooting conditions.Since the above items of information are additional items ofinformation, they (or some of them) may not be set.

The acceptance section 21 accepts image data and checks whether theimage data contains color space information. If the image data does notcontain color space information, the acceptance section 21 supplies theimage data to the color space inferring section 26. If the image datacontains color space information, the acceptance section 21 supplies theimage data to the reliability judging section 23.

The reliability judging section 23 refers to the color space informationthat is appended to the image data and checks whether the color spaceinformation satisfies a predetermined condition. If the color spaceinformation does not satisfy the predetermined condition, thereliability judging section 23 supplies the image data to the colorspace inferring section 26. If the color space information satisfies thepredetermined condition, the reliability judging section 23 outputs theinformation of the color space indicated by the color space informationthat is appended to the image data.

The processing of the reliability judging section 23 will be describedbelow in more detail. For example, as shown in FIG. 3, in terms offunctions, the reliability judging section 23 is composed of an objectdatabase 31, a preprocessing section 32, an area extracting section 33,a color conversion candidates storage section 34, a color conversionsection 35, and a color judgment section 36.

As shown in FIG. 4, in the object database 31, information A indicatingan object, a characteristic quantity B for image recognition of theobject, and reference color information C for color judgment arecorrelated with each other. If the object is a human face, for example,its characteristic quantity can be entropy, an outline shape, or thelike. The reference color information C is information of a referencecolor that is expressed in a predetermined, specific color space(hereinafter referred to as “reference color space”). The referencecolor information C may be information indicating a saturation range, ahue range, etc., as shown in FIG. 4 or information indicating a targetcolor (e.g., RGB values).

The preprocessing section 32 performs prescribed preprocessing on imagedata as a subject of reliability judgment and supplies a processingresult to the area extracting section 33. More specifically, thepreprocessing may be conversion into luminance information. For example,in this embodiment, assume that it is known that input image data isexpressed in a color space of RGB values. If it is unknown whether theRGB values are sRGB values, adobeRGB values, AppleRGB values, or thelike, luminance information Y of each pixel is calculated according tothe following Equation (1):

Y=0.299R+0.587G+0.114B  (1)

where R, G, and B are RGB values of the image data. The calculatedluminance information is supplied to the area extracting section 33.

The area extracting section 33 compares the luminance informationobtained by the preprocessing with the characteristic quantities of therespective objects stored in the object database 31. If the image datahas a range whose luminance information distribution meets acharacteristic quantity, the area extracting section 33 identifies therange as an area of attention and outputs, as area-of-attention defininginformation, information for defining the area of attention and theinformation indicating the object corresponding to the characteristicquantity thus met. The above processing performed by the preprocessingsection 32 and the area extracting section 33 is processing thatrecognizes whether an object stored in the object database 31 iscontained in the image data and defines its area (i.e., its location inthe image data). Such processing is not limited to the above-describedone and may be any processing as long as it can define an area in theimage data containing an object.

The color conversion section 35 converts the input image data into dataof the reference color space while assuming that the input image data isexpressed in the color space that is indicated by the color spaceinformation appended to the image data, and supplies a conversion resultto the color judgment section 36. The color conversion section 35 usesinformation stored in the color conversion candidates storage section 34in order to perform the conversion. In particular, the color conversioncandidates storage section 34 stores conversion parameters (e.g., aconversion matrix or a look-up table) for each of a plurality ofdifferent color space standards (e.g., sRGB, adobeRGB, AppleRGB, etc.).(Hereafter, each set of conversion parameters stored in the storagesection 34 is referred to as “item of color space candidateinformation.”) Thus, the color conversion section 35 will use the itemof color space candidate information (stored in section 34) thatcorresponds to the color space information appended to the image data inorder to convert the image data into the reference color space. Forexample, if the appended data indicates adobeRGB, the color conversionsection 35 will use the parameters (e.g., matrix or look-up table)stored in storage section 34 for adobeRGB to convert the image data intothe reference color space, which is used in the color judgment section36. The result of conversion is supplied to the color judgment section36.

The color judgment section 36 extracts part of the image data receivedfrom the color conversion section 35 in the area defined by thearea-of-attention defining information that is supplied from the areaextracting section 33, and obtains statistical information by performingcalculation using information relating to the colors of the pixels inthe extracted area. (The term “statistical information” is used becausethis information may be statistical information such as a saturationhistogram, a hue histogram, or an average color.) The color judgmentsection 36 compares the statistical information with the reference colorinformation that is stored in the object database 31 and that iscorrelated with the information indicating the object that is containedin the area-of-attention defining information.

If it is found as a result of the comparison that a prescribed conditionis satisfied, that is, values represented by the statistical informationfall within the saturation, hue, and lightness ranges or the differencebetween its average color and the target color is smaller than aprescribed threshold value, the color judgment section 36 outputs theinformation of the color space that is indicated by the color spaceinformation that is appended to the image data. If the prescribedcondition is not satisfied, the color judgment section 36 judges thatthe appended color space information is not reliable. Then, the colorconversion section 35 supplies the image data to the color spaceinferring section 26 and causes it to perform processing of inferring acolor space.

That is, in the reliability judgment processing, a portion whose colortone is known, for example, a skin color portion (in the case of a humanface), is determined by image recognition processing and the reliabilityof information appended to image data is checked on the basis of whetherthe color of the thus-determined portion coincides with a predeterminedcolor.

In this embodiment, the preprocessing, the color conversion processing,etc., are performed on image data having the original size. Analternative procedure is possible in which processing of reducing thesize of input image data is performed and each of those kinds ofprocessing is performed on reduced image data.

In this embodiment, the reliability judging section 23 and the colorspace inferring section 26 (described later) infer the color space ofinput image data and output a result of the inference. The imageprocessing section 24 receives the information indicating the inferredcolor space from one of the reliability judging section 23 and the colorspace inferring section 26, and performs prescribed image processing onthe image data accepted by the acceptance section 21. The prescribedimage processing is color conversion of the image data, chroma,lightness, and contrast corrections, etc. The image processing section24 may supply the image-processed image data to the output section 25after causing the display section 15 to display image data that has beenproduced by the image processing and then receiving a manipulation ofthe user to the effect that the processing result is proper. The outputsection 25 receives the image-processed image data from the imageprocessing section 24 and outputs it to an external device or the like.

The color space inferring section 26 is approximately the same, inconfiguration, as the reliability judging section 23 and is differentfrom the latter in the operation of the color conversion section 35(35′). As shown in FIG. 5, the color space inferring section 26includes, in terms of functions, an object database 31, a preprocessingsection 32, an area extracting section 33, a color conversion candidatestorage section 34, a color conversion section 35′, and a color judgmentsection 36. In FIG. 5, the sections that can be shared with thereliability judging section 23 are given the same reference symbols asthose of the reliability judging section 23.

As shown in FIG. 4, in the object database 31, information A indicatingan object, a characteristic quantity B for image recognition of theobject, and reference color information C for color judgment arecorrelated with each other. If the object is a human face, for example,its characteristic quantity can be entropy, an outline shape, or thelike. The reference color information C may be information indicating asaturation range, a hue range, etc., as shown in FIG. 4 or informationindicating a target color (e.g., RGB values).

The preprocessing section 32 performs prescribed preprocessing on imagedata as a subject of color space inference and supplies a processingresult to the area extracting section 33. Specifically, thepreprocessing may be conversion into luminance information. For example,in this embodiment, assume that it is known that the input image data isexpressed in a color space of RGB values. If it is unknown whether theRGB values are sRGB values, adobeRGB values, AppleRGB values, or thelike, luminance information Y of each pixel is calculated according toEquation (1) where R, G, and B are RGB values of the image data. Thecalculated luminance information is supplied to the area extractingsection 33.

The area extracting section 33 compares the luminance informationobtained by the preprocessing with the characteristic quantities of therespective objects stored in the object database 31. If the image datahas a range whose luminance information distribution meets acharacteristic quantity, the area extracting section 33 identifies therange as an area of attention and outputs, as area-of-attention defininginformation, information for defining the area of attention and theinformation indicating the object corresponding to the characteristicquantity thus met. The above processing performed by the preprocessingsection 32 and the area extracting section 33 is processing thatrecognizes whether an object stored in the object database 31 iscontained in the image data and defines its area (what is called imagerecognition processing). Such processing is not limited to theabove-described one and may be any processing as long as it can definean area containing an object.

The color conversion section 35′ performs color conversion processing onthe image data that has not been subjected to the preprocessing in orderto convert that image data into the color space indicated by thereference color information stored in the object database 31 using thecolor space conversion parameters indicated by each color spacecandidate information (i.e., each item of color space information)stored in the color conversion candidate storage section 34 as a sourcecolor space.

More specifically, this processing is performed in the following manner.One item of color space candidate information (e.g., the parameters forsRGB, adobeRGB or AppleRGB) is selected from the setting of items ofcolor space candidate information. Color conversion processing isperformed to convert the image data into the reference color space whileassuming that the image data is expressed in the color space indicatedby the selected color space candidate information. A conversion resultis stored in the storage section 12 so as to be correlated with theselected color space candidate information. Then, it is checked whetherthere remain unselected items of color space candidate information inthe color conversion candidate storage section 34. If there remainunselected items of color space candidate information, one of them isselected and color conversion processing is again performed for it.

When all the items of color space candidate information (i.e., each ofsRGB, adobeRGB and AppleRGB in the present example) of the setting havebeen selected (i.e., there remains no unselected color space candidateinformation), the color space inferring section 26 judges that theprocessing has been completed and supplies a notice to that effect tothe color judgment section 36.

Receiving, from the color conversion section 35′, the notice to theeffect that the processing has completed, the color judgment section 36obtains plural items of statistical information by calculating, for eachof the plural color conversion processing results stored in the storagesection 12 as corresponding to the respective items of color spacecandidate information, information relating to colors in the areadefined by the area-of-attention defining information that is suppliedfrom the area extracting section 33. (The term “statistical information”is used because this information may be statistical information such asa saturation histogram, a hue histogram, or an average color.) The colorjudgment section 36 compares each of the plural calculated items ofstatistical information with the reference color information that isstored in the object database 31 so as to be correlated with theinformation indicating the object that is contained in thearea-of-attention defining information.

By making the comparison, the color judgment section 36 findsstatistical information that satisfies a prescribed condition, that is,such statistical information that has values represented by it whichfall within the saturation, hue, and lightness ranges or that has adifference between its average color and the target color which issmaller than a prescribed threshold value. The color judgment section 36outputs the information of the color space that is indicated by thecolor space candidate information corresponding to the statisticalinformation thus found.

That is, in the color space inference processing, a portion whose colortone is known, for example, a skin color portion (in the case of a humanface), is determined by image recognition processing and a color spacein which the original image data is expressed is inferred on the basisof whether the color of the thus-determined portion coincides with apredetermined color.

In the color space inferring section 26, the color space inferenceprocessing may be performed on reduced image data that is obtained byreducing the size of image data as a subject of processing in or beforethe pre-processing.

The operation of this embodiment will be described below. Image datathat has been input through the input/output interface section 13 isstored in the storage section 12 as a subject of processing. The controlsection 11 starts a process shown in FIG. 6 according to programs storedin the storage section 12. First, at step S1 the control section 11checks whether image data as a subject of processing containsinformation indicating a color space. If the image data does not containcolor space information (“no”), at step S2 the control section 11 startsprocessing of the color space inferring section 26 and obtains aninference result of a color space of the image data as the subject ofprocessing. At step S3, the control section 11 causes execution ofprescribed image processing that uses the information of the inferredcolor space. At step S4, the control section 11 outputs, through theinput/output interface section 13, image data produced by the imageprocessing. Then, the process is finished.

If the image data as the subject of processing contains color spaceinformation (“yes” result of step S1), the control section 11 (i.e.,reliability judging section 23) performs processing of judging thereliability of the color space information, and infers a color space.Specifically, the reliability judging section 23 judges at step S5whether the color space information satisfies a prescribed condition. Ifthe prescribed condition is not satisfied (“no”), the process proceedsto step S2 and the processing of the color space inferring section 26 isperformed. If the prescribed condition is satisfied at step S5 (“yes”),at step S6 the color space information itself contained in the imagedata is employed as a color space inference result. Then, the processproceeds to step S3.

That is, in this embodiment, the control section 11 performs prescribedimage processing using color space information that has been judgedreliable by the reliability judging section 23 or inferred by the colorspace inferring section 26, and outputs image-processed image datathrough the input/output interface section 13.

As described above, the use of the color space inferring section 26makes it possible to infer the color space of image data as a subject ofprocessing without the need for making an inquiry of a user. That is, ifno color space information is appended to image data or color spaceinformation is judged unreliable even if it is appended to image data,the color space inferring section 26 infers a color space and the imageprocessing section 24 performs prescribed image processing on the basisof an inference result.

According to a second embodiment, it is possible to infer a color spaceby inquiring of a user instead of inferring a color space withoutinvolvement of human hands. Such an image processing apparatus accordingto the second embodiment of the invention will be described below.

The image processing apparatus according to the second embodiment of theinvention is similar, in configuration, to the image processingapparatus according to the first embodiment shown in FIG. 1 and issomewhat different from the latter in the processing of the controlsection 11. Specifically, as shown in FIG. 7 instead of FIG. 2, programsto be executed by the control section 11 include an acceptance section21, an inquiry processing section 22, a reliability judging section 23,an image processing section 24, and an output section 25.

The inquiry processing section 22 obtains plural color conversionprocessing results corresponding to respective items of color spacecandidate information for input image data on the basis of a setting ofthe items of color space candidate information that is stored in thestorage section 12 in advance, and causes the display section 15 todisplay those color conversion processing results. The inquiryprocessing section 22 causes a user to input, through the manipulationsection 14, information indicating a color conversion processing resultof a correct color conversion. Then, the inquiry processing section 22judges that the color space candidate information corresponding to thethus-identified color conversion processing result indicates the colorspace of the image data concerned, and outputs information indicatingthat color space.

The details of the processing of the inquiry processing section 22 willbe described below while assuming that a setting of items of color spacecandidate information is stored in advance in the storage section 12 andthat, for example, the setting of items of color space candidateinformation is a set of items of information indicating color spacesthat may be the color space of input image data.

First, the inquiry processing section 22 performs reduction processingon input image data and generates reduced image data. Then, the inquiryprocessing section 22 selects one item of color space candidateinformation from the setting of items of color space candidateinformation, and performs prescribed image processing while assumingthat the reduced image data is expressed in the color space that isindicated by the selected color space candidate information. In otherwords, the inquiry processing section 22 performs the prescribed imageprocessing on the reduced image data by using, as a source color space,the color space indicated by the selected color space candidateinformation. The prescribed image processing that is performed here maybe, for example, color conversion processing from the color spaceindicated by the selected color space candidate information into anotherprescribed color space (i.e., a reference color space). As noted above,the color space conversion is performed by a commonly known method usingmatrix operations or look-up tables and will not be described in detail.

A result of the prescribed image processing is stored in the storagesection 12 as inquiry candidate image data in such a manner as to becorrelated with the selected color space candidate information. Theinquiry processing section 22 checks whether the setting of items ofcolor space candidate information includes items of color spacecandidate information that have not been selected yet. If there existsuch items of color space candidate information, the inquiry processingsection 22 selects one of those items of color space candidateinformation and performs the prescribed image processing again for it.

When all the items of color space candidate information of the settinghave been selected (i.e., there remains no unselected color spacecandidate information), the inquiry processing section 22 causes thedisplay section 15 to display images based on the respective inquiryimage data that are stored in the storage section 12.

Then, the inquiry processing section 22 waits for a user's manipulationon the manipulation section 14 for selecting one of the images. Uponselection of one image, the inquiry processing section 22 acquires thecolor space candidate information contained in the inquiry image datacorresponding to the selected image and outputs the information of thecolor space that is indicated by the acquired color space candidateinformation.

That is, the inquiry processing section 22 assumes that input image datais expressed in each of plural assumed color spaces and performs imageprocessing for each case. The inquiry processing section 22 presentsindividual image processing results to a user and causes the user toselect image data that was processed correctly. This embodiment canreduce the processing load because image data as a subject of processingis reduced in advance. However, the reduction processing is notindispensable; the image processing may be performed on image datahaving the original size.

A basic operation of the image processing apparatus according to thissecond embodiment will be described below. Image data that has beeninput through the input/output interface section 13 is stored in thestorage section 12 as a subject of processing. The control section 11starts processing shown in FIG. 8 according to programs stored in thestorage section 12. First, at step S11, the control section 11 checkswhether the image data as the subject of processing contains informationindicating a color space (i.e., color space information). If the imagedata does not contain color space information (“no”), at step S12 thecontrol section 11 starts processing of the inquiry processing section22 and generates reduced image data on the basis of the image data. Atstep S13, the control section 11 assumes that each of plural presetitems of color space candidate information is a source color space ofthe image data and performs color conversion processing into aprescribed color space. Then the control section 11 causes the displaysection 15 to display plural results obtained by the color conversionprocessing to present those to a user (inquiry image data presentationprocessing). At step S14, the control section 11 waits for acceptance ofa selection by the user. Upon acceptance of a selection, at step S15 thecontrol section 11 infers that the color space that is indicated by thecolor space candidate information corresponding to the selected colorconversion processing result is the color space of the input image data.At step S16, the control section 11 performs prescribed image processingusing the information of the inferred color space. At step S17, thecontrol section 11 outputs, through the input/output interface section13, image data that has been produced by the image processing. Then, theprocessing is finished.

At step S14, the user selects the most appropriate conversion result.However, if the list of conversion results being displayed on thedisplay section 15 has no appropriate conversion result, the followingoperation may be performed. The user inputs information that specifies acolor conversion such as a character string indicating a color space ormatrix values to be used in a color conversion. The control section 11then infers a color space on the basis of the information thus input.

If it is judged at step S11 that the image data contains color spaceinformation (“yes”), at step S18 the control section 11 performsprocessing of judging the reliability of that color space informationand thereby judges whether the color space information satisfies aprescribed condition. If the color space information satisfies theprescribed condition (“yes”), at step S19 the control section 11 judgesthat the color space information is reliable and employs the color spaceinformation itself contained in the image data as a color spaceinference result. The control section 11 proceeds to step S16, where thecontrol section 11 causes execution of prescribed image processing thatuses the information of the color space that was inferred at step S19.At step S17, the control section 11 outputs, through the input/outputinterface section 13, image data produced by the image processing. Then,the control section 11 finishes the processing. If it is judged at stepS88 that the color space information does not satisfy the prescribedcondition (“no”), the control section 11 judges that the color space isnot reliable and moves to step S12 to continue the processing describedabove from step S12 through step S17.

The first and second embodiments employ the reliability judgmentprocessing that utilizes image recognition processing to judge thereliability of color space information that is input so as to accompanyimage data. However, the reliability judgment can also be performed byother methods. For example, since it is empirically known thatemployment of the sRGB color space is highly probable if the version(Exif Version) of an Exif data structure as information indicating animage data format is “2.2” or less, the reliability judgment may beperformed in the following manner. If the version of an Exif datastructure is “2.2” or less and the color space information of theappended information is “sRGB,” the color space information is judgedreliable and employed as an inference result as it is. If the version ofan Exif data structure is “2.2” or less but the color space informationof the appended information is not “sRGB,” the color space informationis judged unreliable and processing of the color space inferring section26 or the inquiry processing section 22 is started. If the version of anExif data structure is greater than “2.2,” the reliability may be judgedby another method.

Another reliability judging method is as follows. A date and time whenan image was generated (DateTimeOriginal) and a date and time whendigital data was generated (DateTimeDigitized) are compared with eachother by using tag information relating to the date and time. If theycoincide with each other, it is judged that the image data has not beensubjected to editing or the like and the color space information of theappended information is employed as an inference result as it is. If thedate and time when the image was generated (DateTimeOriginal) and thedate and time when the digital data was generated (DateTimeDigitized) donot coincide with each other, the color space information of theappended information is judged unreliable and processing of the colorspace inferring section 26 or the inquiry processing section 22 isstarted.

A tag relating to user information, such as UserComment, may contain anediting history, because there are pieces of application software inwhich an editing history is set in a tag relating to user information.In view of this, the following reliability judgment method is possible.A color space used is inferred by referring to such a tag, morespecifically, an editing history of color space conversion etc., and aninference result is compared with the color space information of theappended information. If coincidence is found, the color spaceinformation of the appended information is employed as an inferenceresult. If coincidence is not found, the color space information of theappended information is judged unreliable and processing of the colorspace inferring section 26 or the inquiry processing section 22 isstarted.

In the image processing apparatus according to the first and secondembodiments, information indicating an inferred color space may beincorporated into image data as a subject of processing. Morespecifically, the control section 11 (i.e., image processing section 24)incorporates color space information as an inference result intoExif-format image data in the form of a color space tag (ColorSpace).

In the image processing apparatus according to the first and secondembodiments, the following processing may be performed. Color spaceinference results of respective image data as subjects of processing aresubjected to statistical processing, and frequencies of occurrence ofitems of information indicating the color spaces obtained by theinference processing are stored. The stored frequencies of occurrenceare used for later inference processing or image data output processing.More specifically, the control section 11 (i.e., inquiry processingsection 22) selects items of color space candidate information in orderof frequencies of occurrence of color spaces of past inference results,and causes the display section 15 to display, in the order offrequencies of occurrence, processing results obtained by using, assource color spaces, the color spaces indicated by the respectiveselected items of color space candidate information.

Further, in the image processing apparatus according to the firstembodiment, if the color space inferring section 26 has failed ininferring a color space, for example, the area extracting section 33 hasfailed in extracting an area corresponding to an object, usingfrequencies of occurrence as an example of results of the statisticalprocessing, the color space inferring section 26 may output, as a colorspace inference result of image data as a subject of processing, a colorspace having the highest frequency of occurrence among the color spacesof image data that were processed in the past.

The above description is directed to the exemplary case in which thecolor space of image data as a subject of processing is basically an RGBspace such as the sRGB space or the adobeRGB space. However, the imageprocessing apparatus according to the invention is not limited to such acase.

For example, the image processing apparatus according to the embodimentscan also be applied to a case of inferring a print standard such asJapanColor or SWOP for an image that is expressed in a CMYK color space.In this case, the above embodiments can be used by performing aCMYK-to-RGB conversion. The image processing apparatus according to theembodiments can also be used in inferring which of RGB and HSB colorspace representations is used.

Although the above description is directed to the exemplary case of theExif format, the invention can also be applied to cases of other imageformats.

The first and second embodiments may be combined together in thefollowing manner. The inquiry processing section 22 of the secondembodiment is incorporated into the image processing apparatus accordingto the first embodiment. When the color space inferring section 26 hasfailed in inferring a color space, for example, the area extractingsection 33 has failed in extracting an area corresponding to an object,image data is supplied to the inquiry processing section 22 and a colorspace is inferred by inquiring of a user.

The controller (e.g., the control section 21) of the illustratedexemplary embodiments is implemented as one or more programmed generalpurpose computers. It will be appreciated by those skilled in the artthat the controller can be implemented using a single special purposeintegrated circuit (e.g., ASIC) having a main or central processorsection for overall, system-level control, and separate sectionsdedicated to performing various different specific computations,functions and other processes under control of the central processorsection. The controller can be a plurality of separate dedicated orprogrammable integrated or other electronic circuits or devices (e.g.,hardwired electronic or logic circuits such as discrete elementcircuits, or programmable logic devices such as PLDs, PLAs, PALs or thelike). The controller can be implemented using a suitably programmedgeneral purpose computer, e.g., a microprocessor, microcontroller orother processor device (CPU or MPU), either alone or in conjunction withone or more peripheral (e.g., integrated circuit) data and signalprocessing devices. In general, any device or assembly of devices onwhich a finite state machine capable of implementing the proceduresdescribed herein can be used as the controller. A distributed processingarchitecture can be used for maximum data/signal processing capabilityand speed.

While the invention has been described with reference to preferredexemplary embodiments thereof, it is to be understood that the inventionis not limited to the disclosed embodiments or constructions. On thecontrary, the invention is intended to cover various modifications andequivalent arrangements. In addition, while the various elements of thedisclosed invention are shown in various combinations andconfigurations, which are exemplary, other combinations andconfigurations, including more less or only a single element, are alsowithin the spirit and scope of the invention.

1. An image processing apparatus which accesses a database that storesitems of characteristic quantity information to be used for recognizing,in image data, respective objects and items of color information of therespective objects, the characteristic quantity information and thecolor information are correlated with each other for the respectiveobjects, comprising: means for judging whether information indicating acolor space of image data, and which is appended to and input togetherwith the image data, satisfies a prescribed condition; means forperforming image recognition processing on the image data using theitems of characteristic quantity information stored in the database andfor acquiring color information of an object that has been recognized inthe image data by the image recognition processing, if it is judged thatthe information indicating the color space and that is appended to theimage data does not satisfy the prescribed condition; and means forsearching the database to retrieve the color information indicating acolor of the object recognized by the image recognition processing andfor identifying a color space of the image data by comparing theacquired color information with the retrieved color information.
 2. Theimage processing apparatus according to claim 1, further comprising:means for performing statistical processing on identification results ofcolor spaces of the image data that was previously processed; and meansfor performing prescribed processing using a result of the statisticalprocessing.
 3. The image processing apparatus according to claim 1,wherein the acquired color information of the object that has beenrecognized in the image data is acquired by converting the image datainto a reference color space a plurality of times using differentconversion parameters each time, each of the different conversionparameters corresponding to a different color space.
 4. The imageprocessing apparatus according to claim 1, wherein the color informationstored in the database includes at least one or more of: informationindicating a saturation range, information indicating a hue range, andinformation indicating a target color.
 5. The image processing apparatusaccording to claim 2, wherein the result of the statistical processingincludes at least one or more of: a saturation histogram, a huehistogram, and an average color.
 6. An image processing apparatus whichaccesses a database that stores items of characteristic quantityinformation to be used for recognizing, in image data, respectiveobjects and items of color information of the respective objects, thecharacteristic quantity information and the color information arecorrelated with each other for the respective objects, the imageprocessing apparatus comprising: a controller that: (1) judges whetherinformation indicating a color space of image data, and which isappended to and input together with the image data, satisfies aprescribed condition; (2) performs image recognition processing on theimage data using the items of characteristic quantity information storedin the database, and acquires color information of an object that hasbeen recognized in the image data by the image recognition processing,if it is judged that the information indicating the color space and thatis appended to the image data does not satisfy the prescribed condition;and (3) searches the database to retrieve the color informationindicating a color of the object recognized by the image recognitionprocessing, and identifies a color space of the image data by comparingthe acquired color information with the retrieved color information. 7.The image processing apparatus according to claim 6, wherein thecontroller also: (3) performs statistical processing on identificationresults of color spaces of the image data that was previously processed;and (4) performs prescribed processing using a result of the statisticalprocessing.
 8. The image processing apparatus according to claim 6,wherein the controller acquires the color information of the object thathas been recognized in the image data by converting the image data intoa reference color space a plurality of times using different conversionparameters each time, each of the different conversion parameterscorresponding to a different color space.
 9. The image processingapparatus according to claim 6, wherein the color information stored inthe database includes at least one or more of: information indicating asaturation range, information indicating a hue range, and informationindicating a target color.
 10. The image processing apparatus accordingto claim 7, wherein the result of the statistical processing includes atleast one or more of: a saturation histogram, a hue histogram, and anaverage color.
 11. An image processing method using an image processingapparatus which accesses a database that stores items of characteristicquantity information to be used for recognizing, in image data,respective objects and items of color information of the respectiveobjects, the characteristic quantity information and the colorinformation are correlated with each other for the respective objects,comprising the steps of: judging whether information indicating a colorspace of image data, and which is appended to and input together withthe image data, satisfies a prescribed condition; performing imagerecognition processing on the image data using the items ofcharacteristic quantity information stored in the database, andacquiring color information of an object that has been recognized in theimage data by the image recognition processing, if it is judged that theinformation indicating the color space and that is appended to the imagedata does not satisfy the prescribed condition; and searching thedatabase to retrieve the color information indicating a color of theobject recognized by the image recognition processing, and identifying acolor space of the image data by comparing the acquired colorinformation with the retrieved color information.
 12. The imageprocessing method according to claim 11, further comprising: performingstatistical processing on identification results of color spaces of theimage data that was previously processed; and performing prescribedprocessing using a result of the statistical processing.
 13. The imageprocessing method according to claim 11, wherein the acquired colorinformation of the object that has been recognized in the image data isacquired by converting the image data into a reference color space aplurality of times using different conversion parameters each time, eachof the different conversion parameters corresponding to a differentcolor space.
 14. The image processing method according to claim 11,wherein the color information stored in the database includes at leastone or more of: information indicating a saturation range, informationindicating a hue range, and information indicating a target color. 15.The image processing method according to claim 12, wherein the result ofthe statistical processing includes at least one or more of: asaturation histogram, a hue histogram, and an average color.
 16. Animage processing program for causing a computer which accesses adatabase that stores items of characteristic quantity information to beused for recognizing, in image data, respective objects and items ofcolor information of the respective objects, the characteristic quantityinformation and the color information are correlated with each other forthe respective objects, to execute the steps of: judging whetherinformation indicating a color space of image data, and which isappended to and input together with the image data, satisfies aprescribed condition; performing image recognition processing on theimage data using the items of characteristic quantity information storedin the database, and acquiring color information of an object that hasbeen recognized in the image data by the image recognition processing,if it is judged that the information indicating the color space and thatis appended to the image data does not satisfy the prescribed condition;and searching the database to retrieve the color information indicatinga color of the object recognized by the image recognition processing,and identifying a color space of the image data by comparing theacquired color information with the retrieved color information.